Traditional multiple choice testing techniques to assess the extent of a person's knowledge in a subject matter include varying numbers of possible choices that are selectable by one-dimensional, or right/wrong (RW) answers. A typical multiple choice test might include questions with three possible answers, where generally one of such answers can be eliminated by the test subject as incorrect as a matter of first impression. This gives rise to a significant probability that a guess on the remaining answers could result in a correct response. Under this situation, a successful guess would mask the true extent or the state of knowledge of the test subject, as to whether he or she is informed (i.e., confident with a correct response), misinformed (i.e., confident in the response, which response, however, is not correct) or lacking information (i.e., having no information). Accordingly, the traditional multiple choice one-dimensional testing technique is highly ineffectual as a means to measure the true extent of knowledge of the test subject. Despite this significant drawback, the traditional one-dimensional, multiple choice testing techniques are widely used by information-intensive and information-dependent organizations such as banking, insurance, utility companies, educational institutions and governmental agencies.
In response to the foregoing drawback, an information referenced testing technique was developed and pioneered by Dr. James Bruno of the University of California at Los Angeles. Information referenced testing (“IRT”) techniques extract the test subject's information response and confidence associated with it, undertake to reduce guessing, and effect a scoring profile that is resistant to the affects of guessing. IRT test queries are generally represented in a two-dimensional format to incorporate two key components of recognition and confidence as part of the test questions or queries in a multiple choice test. The test example below illustrates the general principle of the IRT technique.    1. How many legs does a cat have?            A. 3        B. 4        C. 5            2. What is the common logarithm of the number 3?            A. 43        B. 47        C. 0.047            3. How many states border the state of New Mexico?            A. 4        B. 5        C. 6            4. The Panama Canal connecting the Atlantic and Pacific Oceans generally runs            A. North-South        B. East-West        C. Northeast-Southwest        
Instructions:
Each question has a predetermined point score of +30 if the answer is correct or −100 if the answer is incorrect.
If you can narrow your selection to two choices and eliminate the other choice as being an answer, mark your answer for the two choices: (A or B), (B or C), (A or C). Your response will be assigned a predetermined point score of +10 if the answer is correct or −100 if the answer is incorrect.
If you do not know the answer, you may choose not to respond, in which case, you will not receive any points.
When certain that an answer is correct, the test subject selects a response from one of the letters A, B, or C, which corresponds to the answer that the subject is confident to be correct. Thus the selection of a single letter indicates a high confidence level in the answer on the part of the test subject. If the response reflects the correct answer, a point score of 30 will be assigned. Thus this point score of 30 represents a score assigned for an informed response. However, if the test subject selects one of the letters A, B, or C, which reflects a confidence in that response, a wrong answer for the response will return a point score of −100. This negative point score marks a state of misinformation, and the subject is misinformed as to the substantive information pertaining to that query.
If the subject chooses not to select any of the letters provided, which indicates that he or she has no knowledge or information to provide a response, a zero point score will be returned.
Thus, with respect to the above sample queries, if the test subject answered the above questions 1-B, 2-? 3-BC, and 4-B the test subject would be considered as being informed, uninformed, part informed, and misinformed respectively on these test queries.
As illustrated above, the point scoring system of the IRT for correct and incorrect responses for the confidence levels reflected in the subject's answers are (a) +30, −100 when confident or sure; (b) +10, −100 when between two choices; and (c) 0,0 when the subject is without any information. Under the IRT protocol, a maximum score would be achieved if and only if the test subject is “honest” and does not overvalue his or her information. Thus any incentives to guess at an answer are significantly discounted.
Currently, use of the IRT techniques are on a case-by-case, or batch processing, with test creation, administration, scoring and reporting, which use requires significant human interface, labor and logistic support. More significantly is that informational or education material databases are generally disassociated with the results of the test performance and results interpretation thus impeding remedial actions to reeducate or retrain. The affects of such case-by-case application are further amplified where the tests are to be conducted at various locations.
Accordingly, there is a need for a robust and easily managed integral knowledge assessment and learning system, which is deployable in a distributed computer environment for developing and administering knowledge assessment across chronological and geographical bounds. Such a networked testing system would eliminate batch IRT processing, provide for a wider distribution of test subjects in organizations, ensure full confidentiality of the employee, and allow a more detailed and intelligent learning system which is geared toward the true information need of the user.